Image Quality Measures for Evaluating Gamut Mapping

In this paper we compare different image quality measures for the gamut mapping problem, and validate them using psychovisual data from four recent gamut mapping studies. The psychovisual data are choice data of the form: given an original image and two images obtained by applying different gamut mapping algorithms, an observer chooses the one that reproduces the original image better in his/her opinion. The scoring function used to validate the quality measures is the hit rate, i.e., the percentage of correct choice predictions on data from the psycho-visual tests. We also propose a new image quality measure based on the difference in color and local contrast. This measure compares well to the measures from the literature on our psycho-visual data. Some of these measures predict the observer’s preferences equally well as scaling methods like Thurstone’s method or conjoint analysis that are used to evaluate the psycho-visual tests. This is remarkable in the sense that the scaling methods are based on the experimental data, whereas the quality measures are independent of this data. Introduction Gamut mapping describes how a color image is rendered on a device with limited color reproduction capabilities. This classical problem is still an area of active research – Morovic gives a good recent overview [1]. An important step in improving gamut mapping algorithm is an accurate evaluation of its psycho-visual performance. This is traditionally achieved using psycho-visual tests, where observers have to decide which of the mapped images are the better representation of the original. The data gathered in such a test are typically evaluated using Thurstone’s Law of Comparative Judgement [2]. An alternative approach that we want to evaluate here is to use an image quality measure (independent of observer feedback) to measure the difference of a mapped image to the original. An overview of the state of the art in image quality research can be found for example in [3] or [4]. Image quality measures are successfully used in many imaging applications, such as modeling image distortions, especially in data compression [5]. The advantage of using “good” image quality measures to evaluate gamut mapping algorithms is that they can be used to automatically predict the perceived quality of a mapped image without the need for a new psycho-visual study. Psychovisual tests generally give reliable results for tested settings but the tests are time consuming. Furthermore, an extrapolation to changed settings and new images is problematic. Computing an image quality measure on the other hand provides results immediately. The challenge is to find a measure that correlates well with observers’ preferences. It has to represent the response of the human visual system as a mathematical function. For gamut mapping the main image quality factors are preservation of lightness/color and preservation of spatial details. Artifacts introduced by the mapping algorithms may also be a factor which, however, will be neglected in the present study. Some factors encountered in other image quality applications such as noise or compression artifacts are of minor importance for gamut mapping. The main topic of this paper is a quantitative comparison of the performance of image quality measures with data driven quality measures from psycho-visual tests. The performance of the measures is assessed as the percentage of correctly predicted observer choices on data from a psycho-visual test. The data used to compute these percentages were neither used for data evaluation nor for the optimization of the corresponding measures (compare Section ”Validating the quality measures”). Correlations of psycho-visual gamut mapping evaluation and image quality measures have been published before in [6], where only a general ranking of gamut mapping algorithms has been discussed. Our focus here is on predicting observers’ choices in individual comparisons between mapped images. The remainder of this paper is organized as follows: In the next two sections we describe the image quality measures considered in this paper. Then Thurstone’s method and an extension to conjoint analysis are briefly described as methods for evaluating psycho-visual test data. In the subsequent section we describe how to validate the different image quality measures for gamut mapping. The data sets which we used for validation are described in a section on its own. Finally, we discuss the experimental validation results on data sets and conclude the paper. Image quality measures In this section we review the image quality measures that we have compared. We always compare two images X and Y with n×m pixels. At the pixels xi j ∈ X and yi j ∈ Y , respectively, we consider color coordinates. Mostly we are using the lightness coordinate L in CIELAB color space. If not stated otherwise we do not distinguish in our notation between a pixel and the color coordinate considered at this pixel. Structural Similarity Index (SSIM) The Structural Similarity Index was introduced by Wang et. al. [7] and is defined on quadratic image patches of size k× k at the same location within image X and Y . We computed SSIM for the L coordinate in CIELAB color space. Let PX ⊂ X be such a patch and PY the corresponding patch for Y . We compute the following quantities for the patches: P̄X = 1 k2 ∑ x∈PX x, P̄Y = 1 k2 ∑ y∈PY y, 17th Color Imaging Conference Final Program and Proceedings 21 Figure 1: The original image (on the left) and two gamut mapped images (in the middle and on the right). For the image in middle we have QΔE = 24.65 and QΔLC = 0.341 using HPminDE without detail enhancement, and on the right we have QΔE = 27.00 and QΔLC = 0.318 using HPminDE with details enhancement. Note for the image in the middle QΔE is smaller than for the image on the right, but the middle image has lost a lot of details and has the larger perceptual distance from the original (left image). σPX 2 = 1 k2 −1 ∑ x∈PX (x− P̄X ) , σPY 2 = 1 k2 −1 ∑ y∈PY (y− P̄Y ) , and σPX PY = 1 k2 −1 k2 ∑ i=1 (xi − P̄X )(yi − P̄Y ) The Structural Similarity Index is then defined as SSIM(PX ,PY ) = (2P̄X P̄Y +c1) (2σPX PY +c2) ( P̄2 X + P̄ 2 Y +c1 )( σ2 PX +σ 2 PY +c2 ) , with two constants c1 and c2. As proposed by Wang et. al. [5] we used c1 = 1 and c2 = 9 for these constants and k = 8 for the patch size. From the Structural Similarity Index the image quality measure QSSIM(X ,Y ) can be defined as the Structural Similarity Index SSIM averaged over all possible k× k patches in the images X and Y . The resulting measure is in the range [−1,1], and the higher the QSSIM value, the more similar are the compared images. Laplacian mean square error (LMSE) Like the Structural Similarity Index the Laplacian Mean Square Error (compare [8]) is a local measure for the difference in two images. We compute the following quantities at each pixel (more exactly at L coordinate in CIELAB color space of each pixel, with indices 2 ≤ i ≤ n−1 and 2 ≤ j ≤ m−1) of X and Y , respectively: L(xi j) = x(i+1) j +x(i−1) j +xi( j+1) +xi( j−1)−4xi j and L(yi j) = y(i+1) j +y(i−1) j +yi( j+1) +yi( j−1)−4yi j The image quality measure QLMSE is then defined as QLMSE(X ,Y ) = 1 (n−2)(m−2) n−1 ∑ i=2 m−1 ∑ j=2 ( L(xi j)−L(yi j) )2 . Mean square error (MSE) We also consider the mean square error which is just the squared pointwise difference between the images X and Y . The corresponding image quality measure QMSE is defined as QMSE(X ,Y ) = 1 nm n ∑ i=1 m ∑ j=1 (xi j −yi j), where xi j and yi j are L coordinates in the CIELAB color space for the points in images X and Y respectively. Discrete wavelet transform (DWT) The discrete wavelet transform image quality measure has been defined in [9]. Images X and Y are compared as follows: a discrete wavelet transform is applied to the luminance layer of image X and Y , respectively. Let M f X be the magnitudes of the discrete wavelet transform coefficients obtained for X and frequency band f , and let M f Y be the corresponding magnitudes for image Y . From M f X and M f Y the absolute values of differences d f i (X ,Y ) = ∣∣M f X i −M f Y i ∣∣ , i = 1, . . . , ∣∣M f X ∣∣= ∣∣M f Y ∣∣ . are computed for each frequency band. Let σf (X ,Y ) be the standard deviation of the differences d f i (X ,Y ) for frequency band f . Now, the QDWT(X ,Y ) image quality measure is defined as the mean of the σ f (X ,Y ) for all the frequency bands. In our implementation we use Daubechies’ filter [10] to compute the discrete wavelet transform image quality measure QDWT. 22 ©2009 Society for Imaging Science and Technology A new quality measure The quality measure that we are going to describe in this section is based on the observation that important factors determining the quality of gamut mapping algorithms are color preservation and contrast (detail) preservation. We estimate the degree of color preservation by using the CIELAB ΔE distance measure [11]. The example in Figure 1 demonstrates that QΔE , i.e., the quality measure derived from ΔE, alone is not an accurate quality measure since it neglects the preservation of details. To account also for detail preservation we introduce a contrast preserving measure that we call QΔLC . Our quality measure is then a linear combination of QΔE and QΔLC . Below we describe the two measures QΔE and QΔLC in more detail. The measure QΔE ΔE is defined as the Euclidean distance in CIELAB color space between corresponding pixels in two images X and Y of size n×m. That is, locally at pixel x ∈ X and the corresponding pixel y ∈Y the ΔE distance is defined as: ΔE(x,y) = √ ((Lx −Ly)2 +(ax −ay)2 +(bx −by)2) As our image quality measure QΔE we take the average ΔE over the pixels of the two images, i.e.,